Device for producing multimedia presentation

ABSTRACT

A device for producing a multimedia presentation consisting of content includes a memory, a setting unit, an extraction unit, a requesting unit, a receiving unit, and a combiner. The memory stores first digital media classified by time of creation. The setting unit sets the duration of the presentation and a time period for the content. The extraction unit extracts from the memory portions of the first digital media created during the time period. The requesting unit requests general digital media corresponding to the time period from a general digital media source. The receiving unit receives the general digital media from the general digital media source. The combiner combines the received general digital media with the extracted portions of the first digital media to produce the presentation.

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims priority from Japanese ApplicationNo. 2001-274818 filed Sep. 11, 2001, the disclosure of which is herebyincorporated by reference herein.

BACKGROUND OF THE INVENTION

[0002] The present invention relates to a device for producing amultimedia presentation from stored audio/video data.

[0003] There has been known a software application for producing amultimedia presentation from stored audio/video data. When the softwareis run on a personal computer, the computer displays an iconrepresenting the software. The user uses a mouse to “drag” iconsrepresenting audio/video files into the icon of the software. Thesoftware then extracts random sections of the data in the audio/videofiles and combines the sections together into a multi-mediapresentation.

[0004] However, the video clips of the presentation are arranged in arandom manner, so the presentation lacks a sense of time. Also, theaudio/video files contain only a limited amount of information, mainlyof a nature that is personal to the user. Therefore, the resultingpresentation has a very narrow scope.

SUMMARY OF THE INVENTION

[0005] In order to achieve this objective, a device for producing amultimedia presentation consisting of content includes a memory, asetting unit, an extraction unit, a requesting unit, a receiving unit,and a combiner. The memory stores first digital media classified by timeof creation. The setting unit is operable to set a duration of thepresentation and a time period for the content. The extraction unit isoperable to extract from the memory portions of the first digital mediacreated during the time period. The requesting unit is operable torequest general digital media corresponding to the time period from ageneral digital media source. The receiving unit is operable to receivethe general digital media from the general digital media source. Thecombiner combines the received general digital media with the extractedportions of the first digital media to produce the presentation.

[0006] With this configuration, the presentation includes first digitalmedia synchronized with general digital media of the same time period.The presentation is generated fully automatically based on a fewsettings which can be set by the user.

[0007] It is desirable for the first digital media to be classifiedbased on metadata embedded in the first digital media, the metadataincluding a time stamp indicating when the first digital media wascreated. Such a time stamp is available for most digitally createdcontents, so synchronization is convenient.

[0008] It is desirable that the metadata include location data embeddedin the first digital media indicating where the first digital media wascreated, and that the extraction unit be operable to extract from thememory first digital media corresponding to the location data. With thisconfiguration, the presentation shows more detail and background aboutthe first digital media.

[0009] It is desirable that one of the extracted portions of the firstdigital media and the received general digital media includes music, andthat the combiner combine the extracted portions of the first digitalmedia and the received general digital media in clips of lengthdependent on a tempo of the music. With this configuration, visual andaudio portions of the presentation are harmonized with the same style.

[0010] It is desirable that the extraction unit determine the relativedistribution of the first digital media in the memory among a pluralityof sub-periods of the time period and extract a greater amount of thefirst digital media for ones of the sub-periods having a greater totalamount of the first digital media than others of the sub-periods havinga lesser total amount of the first digital media. With thisconfiguration, the presentation will more accurately reflect activeperiods when more of the first digital media was created, and will havea better balance.

[0011] It is desirable that the setting unit set a ratio of the firstdigital media to the general digital media based on an input from auser. With this configuration, the user can better modify the tone ofthe presentation.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other objects, features and advantages of theinvention will become more apparent from reading the followingdescription of the embodiment taken in connection with the accompanyingdrawings in which:

[0013]FIG. 1 is a block diagram showing a system according to anembodiment of the present invention;

[0014]FIG. 2 is a block diagram showing internal components of acomputing unit and digital camera of the system of FIG. 1;

[0015]FIG. 3 is a flowchart representing operations performed by thecomputing unit to produce a multimedia presentation;

[0016]FIG. 4 is a flowchart representing a presentation producingroutine performed by the computing unit;

[0017]FIG. 5 is a flowchart representing a personal media routineperformed by the computing unit;

[0018]FIG. 6 is a bar graph representing a time-wise distribution ofpersonal media;

[0019]FIG. 7 is a view showing an example of an XML request sent by thecomputing unit to a general media server;

[0020]FIG. 8 is a view showing an example of a DTD document defining theXML of FIG. 7; and

[0021]FIG. 9 is a block diagram showing a modification of theembodiment.

DETAILED DESCRIPTION

[0022] Next, a system according to an embodiment of the presentinvention for producing a multimedia presentation will be described withreference to the attached drawings.

[0023] As shown in FIG. 1, the system 1 according to the embodimentincludes a computing unit 30, a general media server 200, and contentpartners 220, 230, 240. The computing unit 30 is connected to a digitalvideo camera 10, an input unit 120, and a display 130. The computingunit 30 is also connected to other personal networked devices 25 througha home network 20 and to the general media server through a publicnetwork 140, such as the Internet. Examples of other personal networkeddevices 25 include a minidisk recorder/player, a DVD player, avideo-on-demand player, and an MP3 player. The general media server 200is connected to the content partners 220, 230, and 240 through a network210.

[0024] As shown in FIG. 2, the digital camera 10 includes an imagepick-up unit 11, a central processing unit 12, a memory 13, a clock 14,an interface 15, and an audio pick-up unit 16. The image pick-up unit 11picks up images and converts them into digital video data usingwell-known processes. At the same time, the audio pick-up unit 16 picksup sounds and converts them into digital audio data using well-knownprocesses. The digital audio data and the digital video data ismultiplexed by a multiplexer (not shown) into what will be referred tocollectively hereinafter as personal media. The central processing unit12 embeds metadata into the personal media data. The metadata includesthe time that the personal media data was created based on a clocksignal from the clock 14. The central processing unit 12 then stores thepersonal media data in the memory 13.

[0025] The computing unit 30 includes a media data exchange interfacecontroller 40, a memory 50, a network controller 60, a processor 70, agraphics controller 80, an input device controller 90, a hard diskcontroller 100, and a hard disk 110. The hard disk 110 stores softwarefor retrieving and transmitting various data across the networks 20, 140and retrieving personal media data from the digital camera 10. The harddisk 110 also stores software for performing a presentation producingroutine to be described later with reference to the flowchart of FIG. 4.

[0026] The computing unit 30 includes a calendar function and an e-mailfunction. The calendar function displays a calendar on the display 130.The user can use the input unit 120 to input meeting times, specialevents, and the like in the days of the displayed calendar. The calendarfunction also enables the user to highlight special calendar entriesusing a flag or priority indicator. The e-mail function enables exchangeof e-mail messages with other computing devices (not shown) connected tothe network 140. The e-mail function also enables the user to highlightspecial received or sent e-mail messages using a flag or priorityindicator.

[0027] Here, operations performed by the computing unit 30 to produce amultimedia presentation will be described while referring to theflowcharts in FIGS. 3 to 5. The user uses the input unit 120 to starttransfer of the personal digital media, including the embedded metadata,from the memory 13 through operations of the CPU 12, the interface 15,the media data exchange interface controller 40, and the processor 70(S1). The personal digital media is stored on the hard disk 110 throughoperations of the processor 70 and the hard disk controller 100 (S2).After the personal media data is stored, it can be edited usingfunctions of the computing unit 30 as desired.

[0028] Next, the settings for the multimedia presentation are set basedon input from the input unit 120 (S3). The settings include the lengthof the presentation, the time period, the types of general media data tobe included, types of optional personal information, ratio of personalto general media, whether to include or exclude folders, and style ofpresentation.

[0029] The length of the presentation is merely how long the user wishesthe presentation to be, for example, one minute, three minutes, 10minutes, or longer.

[0030] The time period is the time frame from which personal and generalmedia data is to be collected. For example, the user can specify aparticular day, week, or month, or dates or even times that the timeperiod starts and ends.

[0031] Various types of general media can be retrieved from contentpartners 220, 230, and 240 through cooperation with the general mediaserver 200. Examples of possible general media include weatherinformation, news highlights, local news, scenes from top-ten movies,sections of top-ten popular music, and even popular televisioncommercials.

[0032] Optional personal information can be added to the personal mediato enhance the personal portion of the presentation. For example, theentries highlighted using the personal calendar function or messageshighlighted using the e-mail function of the computing unit 30 can beincluded in the presentation to show events that were of particularpersonal importance to the user, which makes the presentation morepersonalized. Another source of optional personal information is theother networked devices 25 connected to the computing unit 30 throughthe network 20. For example, a video-on-demand player can providestatistical information about which videos the user views mostfrequently during the time period. Similarly, an MP3 player can providestatistical information about which songs the user listens to mostfrequently.

[0033] The ratio of personal to general media is a value set by the userto designate how much of the presentation will be personal media and howmuch will be general media from external sources such as the contentpartners.

[0034] Folders representing information files can be selectively shownin the presentation.

[0035] The style of presentation is determined automatically by musicselected either as part of the general media or the optional personalinformation. Basically, the clip length of the video portion of thepresentation is dependent on the tempo of the accompanying music. Forexample, when the general media includes a top-ten pop song with a quicktempo, then the clip length is set relatively short, resulting in afast-paced presentation. On the other hand, if the optional personalinformation includes a classical song with a slow tempo, then the cliplength is set relatively long, resulting in a more relaxed presentation.If neither the general media nor the optional personal informationincludes music, then the user must set the style of presentationmanually.

[0036] Next, a presentation producing routine is performed (S4). Duringthe presentation producing routine, the computing unit 30 organizesprivate and general media according to the user's settings. Thepresentation producing routine will be described with reference to theflowchart in FIG. 4.

[0037] When the presentation producing routine is started, the length ofthe total general media is determined (S10). The length of the totalgeneral media is the product of the presentation length times the ratioof personal to general media (length of total general media=presentationlength×ratio of personal to general media).

[0038] Next, a personal media routine is performed (S11). Here, thepersonal media routine will be described with reference to the flowchartin FIG. 5. First, the total length of the personal media portion of thepresentation is determined by subtracting the length of the generalmedia portion from the presentation length (personal media=presentationlength−general media) (S20). Then amounts of the personal media to beextracted for each sub-period of the overall presentation time periodare determined based on the distribution of personal media over time(S21). FIG. 6 shows an example for distribution of personal media over aone year period from January to December. In the example, there is atotal of 30 hours of personal media for the year, with eight hours of itconcentrated in March. Therefore, if the time period of the presentationwere set to this one year period and the presentation length were set to15 minutes, then four minutes worth of personal media would be extractedfrom March content.

[0039] Next, the length of personal media sections is determined basedon the setting of the presentation style, that is, short length forrapid presentation style and longer length for slower presentation style(S22). Then sections of the personal media data are selected at random(S23).

[0040] Next, the length and position of general media sections in thepresentation is determined (S12). In the same manner as with personalmedia, the average length of general media sections for the presentationis based on style of presentation. Also, the frequency of general mediasections depends on the time-based distribution of personal media. Inthe example of FIG. 6, March has eight times more personal media thanSeptember or October, so eight times more general media from March willbe added to the presentation than from September or October. Also, nogeneral media will be provided for August.

[0041] Next, what optional personal information is to be included in thepresentation is determined (S13). Most of the information gathered fromthe e-mail function and the PC calendar function is text, so thisinformation will be animated and laid over the video picture in thepre-defined style of the video. Highlighted e-mail messages that containan image or some other attachment, and movie clips based on usagestatistics, are included as part of the personal media portion of thepresentation.

[0042] Then, the general media is downloaded from the general mediaserver 200 (S14). The general media server 200 stores all of the generalmedia provided by the contents partners 220, 230, 240 and updates thedata for the general media regularly. When producing a presentation, thecomputing unit 30 sends a request to the general media server 200 fortypes of general media set in S3 in amounts determined in S12. Forexample, if the user designates that world-wide news is the only generalmedia to be included in the presentation, then the computing unit 30will download the information from the general media server 200 bysending such a request to the general media server 200. The request fromthe computing unit 30 includes the presentation time period andsub-periods, and flexibility in the length of the general information.An example request sent using extensible markup language (XML) is shownin FIG. 7. The XML is defined by the document type definition (DTD)shown in FIG. 8. The computing unit 30 and the general media server 200both understand the same DTD and so can communicate using XML messages.A content partner that provides movies or music as general media willallow downloading of authorized highlight scenes or tracks. A specialinterface is required to enable this kind of downloading.

[0043] The downloaded general media is saved on the hard disk 110 (S15).Then, a presentation script is generated by combining the personal mediaselected in S23 and the general media stored in S15 (S16). Thepresentation is played in real time as produced (S5).

[0044]FIG. 9 shows a modification of the embodiment. In themodification, the computing unit 30 is connected directly to the contentpartners 220, 230, 240 through the network 140. The general media server200 provides the computing unit 30 with a directory including linkinginformation for connecting with the content partners 220, 230, 240 toobtain general media. The general media server 200 provides thecomputing unit 30 with regular updates of the link directory. With thisscenario, the content partners 220, 230, 240 need to establish aninterface that is understood by the computing unit 30.

[0045] While the invention has been described in detail with referenceto specific embodiments thereof, it will be apparent to those skilled inthe art that various changes and modifications may be made thereinwithout departing from the spirit of the invention, the scope of whichis defined by the attached claims.

[0046] For example, the embodiment describes personal digital mediabeing provided by a digital video camera 10. However, the digital mediacould be provided from any source, such as the computing unit 30 itself.

[0047] Also, the embodiment describes various sources of usagestatistics, such as an MP3 player and a video-on-demand player, as beingexternal to and networked to the computing unit 30. However, thecomputing unit 30 itself can serve as a source of user statistics. Forexample, the computing unit 30 can be provided with an MP3 playerfunction and a video-on-demand function that communicate with thepresentation producing module to provide user statistics.

[0048] The ratio of personal to general media is described as a valuethat can be freely set by the user. However, the ratio can be a fixedvalue. This would be beneficial when the computing unit 30 is aninteractive television, which emphasizes convenience to the user overflexibility. Further, the ratio could be dependent on the number ofsources of general information for the presentation, which is a numberdecided by the user. For example, if the user selects weather-details,international news, movie highlights, and sport-highlights for soccer,baseball and tennis, then the ratio of personal to general media will beset to a lower value than if the user only selects weather. That is, thepercentage of general information will be higher in the former case thanin the latter case.

[0049] Although in the embodiment, the time-wise distribution ofpersonal media is based on month-long sub-periods, the sub-periods couldbe any length, such as a day or even hours, and the length of thesub-periods could vary depending on the scope of the presentation timeperiod. Also, although the clips of personal media are selected randomlyin the embodiment, high level content analysis tools could be providedto make selection more specific.

[0050] The digital camera 10 can be provided with a global positioningsystem (GPS) and incorporate a position stamp into the personal media inaddition to the time stamp. In this case, based on the position stamp inthe personal media, the request from the computing unit 30 can include arequest for news, weather reports, and the like from the area where thepersonal media was originally created.

[0051] Instead of playing the presentation as it is produced, thepresentation could be stored in memory for use at a later time.

1. A device for producing a multimedia presentation consisting ofcontent, comprising: a memory storing first digital media classified bytime of creation; a setting unit operable to set a duration of thepresentation and a time period for the content; an extraction unitoperable to extract from the memory portions of the first digital mediacreated during the time period; a requesting unit operable to requestgeneral digital media corresponding to the time period from a generaldigital media source; a receiving unit operable to receive the generaldigital media from the general digital media source; and a combineroperable to combine the received general digital media with theextracted portions of the first digital media to produce thepresentation.
 2. The device as claimed in claim 1, wherein the firstdigital media is classified based on metadata embedded in the firstdigital media, the metadata including a time stamp indicating when thefirst digital media was created.
 3. The device as claimed in claim 2,wherein the metadata includes location data indicating where the firstdigital media was created, and the extraction unit is operable toextract from the memory first digital media corresponding to thelocation data.
 4. The device as claimed in claim 1, wherein one of theextracted portions of the first digital media and the received generaldigital media includes music, the combiner combining the extractedportions of the first digital media and the received general digitalmedia in clips of length dependent on a tempo of the music.
 5. Thedevice as claimed in claim 1, wherein the extraction unit determines therelative distribution of the first digital media in the memory among aplurality of sub-periods of the time period and extracts a greateramount of the first digital media for ones of the sub-periods having agreater total amount of the first digital media than others of thesub-periods having a lesser total amount of the first digital media. 6.The device as claimed in claim 1, wherein the setting unit sets a ratioof the first digital media to the general digital media based on aninput from a user.